Papers with QA system
FeTaQA: Free-form Table Question Answering (2022.tacl-1)
Copied to clipboard
Linyong Nan, Chiachun Hsieh, Ziming Mao, Xi Victoria Lin, Neha Verma, Rui Zhang, Wojciech Kryściński, Hailey Schoelkopf, Riley Kong, Xiangru Tang, Mutethia Mutuma, Ben Rosand, Isabel Trindade, Renusree Bandaru, Jacob Cunningham, Caiming Xiong, Dragomir Radev, Dragomir Radev
| Challenge: | Existing table-based question answering datasets lack advanced information-based questions that require reasoning and integration of information pieces retrieved from structured knowledge sources. |
| Approach: | They propose a dataset with 10K Wikipedia-based table, question, free-form answer, supporting table cells pairs that can be used to generate an answer. |
| Outcome: | The proposed dataset has 10K Wikipedia-based table, question, free-form answer, supporting table cells pairs. |
PDFTriage: Question Answering over Long, Structured Documents (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing approaches to document QA use a pre-retrieval step to retrieve the relevant context from documents, but this is incongruous with the user's mental model of the document. |
| Approach: | They propose an approach called PDFTriage that enables models to retrieve the context based on either structure or content. |
| Outcome: | The proposed approach can retrieve context based on structure or content across several classes of questions where existing retrieval-augmented LLMs fail. |
Entailment Tree Explanations via Iterative Retrieval-Generation Reasoner (2022.findings-naacl)
Copied to clipboard
Danilo Neves Ribeiro, Shen Wang, Xiaofei Ma, Rui Dong, Xiaokai Wei, Henghui Zhu, Xinchi Chen, Peng Xu, Zhiheng Huang, Andrew Arnold, Dan Roth
| Challenge: | Large language models have achieved high performance on various natural language benchmarks, but the explainability of their output remains elusive. |
| Approach: | They propose an architecture called iterative retrieval-generation reasoner that generates an entailment tree that explains a given hypothesis by using premises from C. |
| Outcome: | The proposed model outperforms existing benchmarks on premise retrieval and entailment tree generation with around 300% gain in overall correctness. |
RAG4ITOps: A Supervised Fine-Tunable and Comprehensive RAG Framework for IT Operations and Maintenance (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have improved the open-domain QA’s performance, but how to efficiently handle enterprise-exclusive corpora and build domain-specific QA systems are still not studied for industrial applications. |
| Approach: | They propose a general and comprehensive framework based on Retrieval Augmented Generation (RAG) and facilitate the whole business process of establishing QA systems for IT operations and maintenance. |
| Outcome: | The proposed framework achieves superior results on two kinds of QA tasks. |
Using Interactive Feedback to Improve the Accuracy and Explainability of Question Answering Systems Post-Deployment (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing work on question answering focuses on the pre-deployment stage; building an accurate model for deployment. |
| Approach: | They collect feedback from users and train a neural model with the feedback data. |
| Outcome: | The proposed model can explain the correctness or incorrectness of an answer. |
End-to-End Beam Retrieval for Multi-Hop Question Answering (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing beam retrieval frameworks for multi-hop question answering were customized for two-hop questions and were poorly supervised. |
| Approach: | They propose an end-to-end beam retrieval framework for multi-hop question answering . they combine an encoder and two classification heads to optimize the retrieval process . |
| Outcome: | The proposed framework improves on MuSiQue-Ans and surpasses all previous retrievers on HotpotQA and achieves 99.9% precision on 2WikiMultiHopQA. |
Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for open-domain question-answering use an open book approach . a recent alternative is to retrieve from a collection of previously-generated question-annwer pairs . |
| Approach: | They propose a new QA system that augments a text-to-text model with a large memory of question-answer pairs and a task for the latent step of question retrieval. |
| Outcome: | The proposed system outperforms closed-book QA and can answer multi-hop questions. |
CMQA: A Dataset of Conditional Question Answering with Multiple-Span Answers (2022.coling-1)
Copied to clipboard
| Challenge: | Existing QA datasets only contain unconditional and parallel answers . conditional question answering with hierarchical multi-span answers is challenging for the community to solve . |
| Approach: | They propose a conditional question answering task with hierarchical multi-span answers . they propose CMQA, which contains conditional and hierarchic samples . |
| Outcome: | The proposed task can be used to build more reliable and sophisticated QA systems. |
Calibrating Trust of Multi-Hop Question Answering Systems with Decompositional Probes (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work in multi-hop QA has shown that performance can be boosted by decomposing questions into simpler, single-hop questions. |
| Approach: | They propose to decompose multi-hop questions into simpler, single-hop ones to create explanations by probing a neural QA model with them. |
| Outcome: | The proposed approach can be used to generate explanations by probing a neural QA model with them. |
Learning a Cost-Effective Annotation Policy for Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | State-of-the-art question answering systems require large amounts of training data for which labeling is time consuming and thus expensive. |
| Approach: | They propose a framework for annotating QA datasets that entails learning a cost-effective annotation policy and a semi-supervised annotation scheme. |
| Outcome: | The proposed approach can reduce up to 21.1% of the annotation cost compared with traditional methods . the proposed approach is based on a cost-effective annotation policy and semi-supervised annotation scheme . |
NoiseQA: Challenge Set Evaluation for User-Centric Question Answering (2021.eacl-main)
Copied to clipboard
| Challenge: | Question-Answering (QA) systems are deployed in the real world . a lack of research attention has been devoted to studying the issues that arise when people use QA systems. |
| Approach: | They show that component components that precede an answering engine can introduce varied and considerable sources of error. |
| Outcome: | The proposed evaluations highlight the need for QA evaluation to expand to consider real-world use. |
Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering (D19-1)
Copied to clipboard
| Challenge: | Arras et al., 2017) suggest an unsupervised strategy for the selection of justification sentences for multi-hop question answering (QA) . |
| Approach: | They propose an unsupervised strategy for the selection of justification sentences for multi-hop question answering that maximizes the relevance of the selected sentences, minimizes overlap between selected facts, and maximizes coverage of both question and answer. |
| Outcome: | The proposed strategy improves state-of-the-art supervised QA model on two multi-hop QA datasets: AI2’s Reasoning Challenge (ARC) and Multi-Sentence Reading Comprehension (MultiRC). |
Neural Natural Logic Inference for Interpretable Question Answering (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing question answering models are based on textual entailment tasks . prior work has focused on QA on premise-based questions . |
| Approach: | They propose a neural-symbolic QA approach that integrates natural logic reasoning within deep learning architectures towards developing effective question answering models. |
| Outcome: | The proposed model outperforms previous work on multiple-choice science questions . it integrates natural logic reasoning within deep learning architectures to build proof paths . |
FocusQA: Open-Domain Question Answering with a Context in Focus (2022.findings-emnlp)
Copied to clipboard
Gianni Barlacchi, Ivano Lauriola, Alessandro Moschitti, Marco Del Tredici, Xiaoyu Shen, Thuy Vu, Bill Byrne, Adrià de Gispert
| Challenge: | a new method for question answering with a context in focus simulates a free interaction with QA systems. |
| Approach: | They introduce question answering with a cotext in focus task that simulates a free interaction with QA systems. |
| Outcome: | The proposed model outperforms state-of-the-art models for question answering with a context in focus up to 21.3% absolute points. |
Semi-supervised Training Data Generation for Multilingual Question Answering (L18-1)
Copied to clipboard
| Challenge: | Existing datasets for question answering (QA) tasks mostly support only English . however, existing resources for these tasks are labor intensive . |
| Approach: | They propose to combine Korean QA datasets with machine-translated English resources to build seed resources. |
| Outcome: | The proposed approach leads to 71.50 F1 on Korean QA (comparable to 77.3 F1) |
A Nil-Aware Answer Extraction Framework for Question Answering (D18-1)
Copied to clipboard
| Challenge: | Recent research suggests that reading comprehension-based question answering systems assume that every question has a valid answer in the associated passage. |
| Approach: | They propose a novel nil-aware answer span extraction framework that can return Nil or a text span from the associated passage as an answer in a single step. |
| Outcome: | The proposed framework outperforms baseline approaches on a newsQA dataset. |
Unsupervised Multi-hop Question Answering by Question Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing training data for multi-hop question answering (QA) is time-consuming and resource-intensive. |
| Approach: | They propose an unsupervised framework that generates human-like multi-hop training data from homogeneous and heterogeneously data sources. |
| Outcome: | The proposed framework achieves 61% and 83% of the supervised learning performance for the HybridQA and HotpotQA datasets. |
Localizing Open-Ontology QA Semantic Parsers in a Day Using Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new toolkit for localizing a semantic parser for a language is proposed . the proposed approach is based on a method for question answering systems . |
| Approach: | They propose a toolkit that leverages Neural Machine Translation systems to localize a semantic parser for a new language. |
| Outcome: | The proposed approach outperforms state-of-the-art methods in 10 new languages . it can be deployed in restaurants and hotels in less than 24 hours . |
Chat or Learn: a Data-Driven Robust Question-Answering System (2020.lrec-1)
Copied to clipboard
| Challenge: | QA systems tend to perform poorly at chitchat, while data-driven chatbots are typically user-friendly but not goal-oriented . |
| Approach: | They propose to use a controller to perform dialogue act classification and feed user input either to a sequence-to-sequence chatbot or to QA systems. |
| Outcome: | The proposed system is a spoken QA application for the Google Home smart speaker. |
Knowledge Graph - Deep Learning: A Case Study in Question Answering in Aviation Safety Domain (2022.lrec-1)
Copied to clipboard
Ankush Agarwal, Raj Gite, Shreya Laddha, Pushpak Bhattacharyya, Satyanarayan Kar, Asif Ekbal, Prabhjit Thind, Rajesh Zele, Ravi Shankar
| Challenge: | Existing Question Answering systems for commercial aviation use a large number of documents . a Knowledge Graph (KG) guided Deep Learning (DL) based system can be used to query the documents based on accident reports . |
| Approach: | They propose a Knowledge Graph (KG) guided Deep Learning (DL) based Question Answering system to cater to these requirements. |
| Outcome: | The proposed system achieves 7% and 40% increase in accuracy over existing systems. |
Unsupervised Question Decomposition for Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing QA systems struggle to answer complex questions because information is scattered in different places. |
| Approach: | They propose an unsupervised algorithm that decomposes hard questions into simpler sub-questions . they propose an algorithm that can be used to generate a final answer from millions of questions . |
| Outcome: | The proposed algorithm decomposes hard questions into simpler sub-questions that existing QA systems can answer. |
Contrastive Domain Adaptation for Question Answering using Limited Text Corpora (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing question generation methods rely on large amounts of synthetically generated datasets and costly computational resources. |
| Approach: | They propose a framework for domain adaptation that combines question generation and domain-invariant learning to answer out-of-domain questions in settings with limited text corpora. |
| Outcome: | The proposed framework improves on state-of-the-art questions in a domain with limited text corpora. |
PolQA: Polish Question Answering Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent proposed systems for open-domain question answering (OpenQA) require large amounts of training data to achieve state-of-the-art performance. |
| Approach: | They propose an efficient annotation strategy that increases passage retrieval accuracy@10 by 10.55 p.p. while reducing the annotation cost by 82%. |
| Outcome: | The proposed approach increases passage retrieval accuracy @10 by 10.55 p.p. while reducing the annotation cost by 82%. |
You Make me Feel like a Natural Question: Training QA Systems on Transformed Trivia Questions (2024.emnlp-main)
Copied to clipboard
Tasnim Kabir, Yoo Yeon Sung, Saptarashmi Bandyopadhyay, Hao Zou, Abhranil Chandra, Jordan Boyd-Graber
| Challenge: | Existing question-answering datasets are expensive and difficult to annotate and time-consuming to gather. |
| Approach: | They propose to transform Manchester questions into web queries using the same question datasets. |
| Outcome: | The proposed questions can be trained on a Manchester QA dataset using the Quiz Bowl (QB) sample. |
Retrieval-based Question Answering with Passage Expansion Using a Knowledge Graph (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in dense neural retrievers and language models have hindered performance, especially for less common entities and facts. |
| Approach: | They propose a multi-modal passage retrieval model that combines entity features and textual data to improve retrieval precision for less common entities. |
| Outcome: | The proposed model improves retrieval precision on less common entities and facts on common benchmarks. |